Complexity Constraints in Two - Armed Bandit Problems : An Example

This paper derives the optimal strategy for a two armed bandit problem under the constraint that the strategy must be implemented by a finite automaton with an exogenously given, small number of states. The idea is to find learning rules for bandit problems that are optimal subject to the constraint that they must be simple. Our main results show that the optimal rule involves an arbitrary initial bias, and random experimentation. We also show that the probability of experimentation need not be monotonically increasing in the discount factor, and that very patient decision makers suffer almost no loss from the complexity constraint.